Performance Versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems

We present the results of comparative analysis performance versus complexity for several types artificial neural networks (NNs) used nonlinear channel equalization in coherent optical communication systems. The comparison has been carried out using an experimental set-up with transmission dominated by Kerr nonlinearity and component imperfections. For first time, we investigate application to convolution layer (CNN) combination a bidirectional long short-term memory (biLSTM) design combining CNN multi-layer perceptron. Their is compared one delivered previously proposed NN equalizer models: biLSTM layer, three-dense-layer perceptron, echo state network. Importantly, all architectures have initially optimized Bayesian optimizer. derivation computational associated each type -- terms real multiplications per symbol so that these can be applied large number demonstrated specific considered system convolutional coupled (CNN+biLSTM) provides highest Q-factor improvement reference linear chromatic dispersion compensation (2.9 dB improvement). examine trade-off between equalizers demonstrate CNN+biLSTM best option when not constrained, while restrict lower levels, three-layer perceptron performance. Our different NNs generic wide range physical engineering